Family of Probability Distributions Derived from Maximal Entropy Principle with 

Scale Invariant Restrictions 
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Using statistical thermodynamics, we derive a general expression of the stationary probability 
distribution for thermodynamic systems driven out of equilibrium by several thermodynamic forces. 
The local equilibrium is defined by imposing the minimum entropy production and the maximum 
entropy principle under the scale invariance restrictions. The obtained probability distribution 
presents a singularity that has immediate physical interpretation in terms of the intermittency 
models. The derived reference probability distribution function is interpreted as time and ensemble 
average of the real physical one. A generic family of stochastic processes describing noise-driven 
intermittency, where the stationary density distribution coincides exactly with the one resulted from 
entropy maximization, is presented. 
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In the Onsager region, the minimum entropy production 
(MEP) theorem provides a variational characterization 
of a stationary states, both for macroscopic systems and 
for stochastic models. However, to characterize some un- 
known events with a statistical model, we should choose 
the one that has maximum entropy (MaxEnt's princi- 
ple). This means that, out of all probability distributions 
consistent with a given set of constraints, we choose the 
one that has maximum uncertainty. Entropy is there- 
fore regarded as a measure of information. The MaxEnt 
principle developed in communication and in information 
technology has recently been found to have a wide rang- 
ing applications in many areas of science. In evolutionary 
biology, the MaxEnt principle is used to explain the ob- 
served species abundance distribution. In this case the 
constraint may be set by the habitat, which fixes the aver- 
age population size of the species [HQ- This principle is 
also applied for describing the activity in a variety of neu- 
ral networks [l]-[l2|, the statistics of amino acid substitu- 
tions in protein families [l3|-[l6| and in material science 
17| and in Bosc-Einstein condensate in a dye microcavity 
18j . It is also proven that the MaxEnt model can provide 
a good description of data from seemingly completely un- 
related phenomena observed in social contexts po| . 
For example, it describes very well the equilibrium dis- 
tribution function for the city-population subject to two 
additional scale-invariant constraints: the normalization 
of the probability distribution function and the expected 
value of the city-population [21|. 

The purpose of this paper is to use statistical thermody- 
namics to derive a reference density distribution function 
(DDF) for open thermodynamic systems close to a local 
equilibrium state. To this end, we assume the validity of 
the minimum entropy production principle and the Max- 



Ent principle on the random variables, under the scale 
invariance restrictions. By definition, the reference DDF 
(indicated with J-" ) is an initial distribution function. 
J-° should depend only on the invariants of motion, with 
the property to evolve slowly from the local equilibrium 
state i.e., it remains confined for sufficiently long time. 
Hence, the reference DDF results in a perturbation of 
the local equilibrium state. We consider open thermo- 
dynamic systems obeying to Prigogine's statistical ther- 
modynamics. We then define the local equilibrium state 
by adopting a minimal number of hypotheses. Finally, 
we link the density distribution function with particle's 
DDF. The density probability distribution of finding a 
state in which the values of the fluctuating thermody- 
namic variable, /3 K , lies between /3 K and fi K + dj3 K is 



T = A/ U cxp[-A / S'] 



(1) 



where No ensures normalization to unity, and we have in- 
troduced the dimensionless (density of) entropy produc- 
tion A/5. The negative sign in Eq. JT]) is due to the fact 
that, during the processes, —A/5 < 0. Indeed, if —A/5 
were positive, the transformation j3 K —¥ j3' K would be a 
spontaneous irreversible change and thus be incompati- 
ble with the assumption that the initial state is a stable 
(local) equilibrium state (22[- We suppose that the sys- 
tem is subject to N thermodynamic forces. The entropy 
production A/ S is linked to the thermodynamic forces 
X K , the thermodynamic fluctuation /3 K , and the thermo- 
dynamic flows J K by the following set of equations [23| 



XK= dAiS j = dp* diS 
df3 K ' K dt ' dt 



N 



^A K J fc >0 (2) 



Our aim is firstly to derive the reference DDF, J 70 , by 
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imposing the MEP and the MaxEnt principle, submit- 
ted to the scale invariance constraints. Successively we 
shall determine the class of stochastic processes whose 
stationary PDF includes J 70 as a special case. 
Let us consider an open thermodynamic system subject 
to N = N + 1 thermodynamic forces. N thermodynamic 
forces are linked to N fluctuations of Prigogine's type 
(i.e., the entropy production is expressed in quadratic 
form with respect to these fluctuations. For an exact 
definition of Prigogine's fluctuations refer to (22[, [24j . 
One thermodynamic force is linked to a fluctuation of a 
different nature. This random variable has the density 
distribution function, which satisfies the MaxEnt prin- 
ciple under the scale invariant restrictions. This fluc- 
tuation will be indicated with w whereas the Prigogine 
fluctuations will be denoted by /3 Kl with k = 1, ■ • • ,,N. 
Hence, $ K = /3k for k = 1, ■ • • , N and Pn+i = w. To be 
more precise, we identify the local equilibrium state by 
imposing the following two conditions. 

i) The local equilibrium state corresponds to the values 
of the fluctuations f3 K with k = 1, • • • , N for which 
the entropy production tends to reach an extreme. 

Under this assumption, close to the local equilibrium, the 
entropy production can be brought into the form 

1 N 

- A/5 = go(w) - - ^ 9nj{w)f3 K ^j + h -°- t - ( 3 ) 

with g (w) = -A/5 \p l ...p N=0 

where h.o.t. stands for higher order terms. Coefficients 
g K j are directly linked to the transport coefficients of the 
system (25[. Therefore, the general expression for T, 
given by Eq. (Q}, becomes the reference DDF, J 70 , when 
the entropy production is provided by Eq. ©. The DDF 
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(4) 



V(w) = Af exp[g (w)} = W cxp[-A/5 \^...p t 

related to the variable w, at f3 K — (with k = 1 ■ ■ ■ N), 
is determined by the following condition. 

ii) At the extremizing values f3 K — with k = 1, • • • , N 
under the scale invariance restrictions, the system 
tends to evolve towards the maximal entropy con- 
figurations. 

Notice that in particular, if the imposed the two scale 
invariant restrictions, E[w] = const. > and E[ln(w)] = 
const, (where E[ ] is the expectation operation), together 
with normalization, the density distribution for the w 
variable is given by a gamma distribution function [26j j 



V{w) = Now 1 - 1 exp(-w/9) 



(5) 



Here, we have introduced the scale parameter 8 and the 
shape parameter 7, which could be determined by the 
fit of experimental data. The goal is now to justify a 
posteriori the result found in Ref. [25j : The kinetic en- 
ergy dependence of the DDF, V{w), at the most probable 



( equilibrium ) values of the remaining phase space vari- 
ables, is expressed by the gamma distribution. Notice 
that if we normalize V(w) to unity then it can be in- 
terpreted as the conditional probability density function 
(PDF), p{w), conditioned by (3 K = with k = 1,---N 
Le.,p{w)=V(w\f3 1 = --- = f3 N =0). 
We consider the entropy 5 [/?(.)] of probability density 
function (PDF), p(w) > 0, given by 



%(•)] = -/ P (w)\og(p(w))dw 



(6) 



We investigate the consequences of the MaxEnt princi- 
ple submitted to the most general scale-invariant restric- 
tions. Let us then start to consider the following restric- 
tions 



w ak p(w)dw = E(u> afc ) = p k ; jfe = 0,l,...,n (7) 



where, at this stage, a k , and p k are real numbers. How- 
ever, for completeness, we should also include the follow- 
ing restriction obtained as the limit case of ([7]) 



E(logH) = v 



(8) 



where v is a real number. Indeed, suppose that we have 
for some fixed k : ^ = e « 1, then 



E(w £ ) = p k 



(9) 



From Eqs ([7]) and by taking into account the normal- 
ization condition on the probability [see the forthcoming 
Eq. (JTTJ)], we get 



E 



(10) 



If the support of the PDF p(w) is concentrated mainly 
on the domain where | log(w)| is not too large, then we 
can approximate: (w £ — u; ) /e = log(w). In this case 
Eq. dTDJ) reduces to Eq.©. Finally, Eq.© can be seen 
as the limit case of and, for the sake of generality, 
it should be taken into account among the equations ex- 
pressing the scale- invariance restrictions. It is worthwhile 
mentioning that the equilibrium distribution function, 
obtained by imposing restriction ([8]) , retrieves the expres- 
sion currently used for fitting the numerical steady-state 
solution of the simulations for the Ion Cyclotron Radia- 
tion Heating (ICRH) FAST-plasmas, and for describing 
various scenarios of tokamak-plasmas [25j|. Another ex- 
ample showing the usefulness of constraint (JSJ can be 
found at the end of this report. Let us now consider the 
following simplest physical case 



p(w)dw = 1 
wp{w)dw = E(u>) = p\ 



(11) 
(12) 
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Hence, n > 1 and, in particular 



Mo = 1; = 0; ai = 1 



(13) 



where p\ — T/m, with T and m are temperature and 
particle's mass, respectively. 

To understand the meaning of Eqs and @, for n = 1, 
we find Eq. from the MaxEnt principle submitted to 
the constraints 0, (JTT|) and (fl2j) . In the following, we 
shall be able to give only a partial answer to this question. 
The restrictions on the PDF given by Eq. and Eq. 
are invariant under scale transformations. This means 
that the effect of the scale transformation w — > kw' on the 
restrictions ([7]) and ([5]) is E(u/ Qfe ) = p ak = p ak k~ ak and 
E(log(u/)) = v' = v— log(fc), respectively. So restrictions 
© and © remain in the same class specified by the 
functions w ak and log(w), respectively. 
Remark (1). It remains to explain the ansatz: n = 1, 
and Eq. ([5]) [naturally combined with restrictions (jTTJ) 
and (fl2)]. 

By denoting with Afc,0<fc<rt + 1 the Lagrange mul- 
tipliers in the problem of maximizing the entropy given 
by Eq. (J6j) , with the restrictions and ((5$, we get 

n 

log = 1 - 5> feW Qfc - A„+i logH (14) 

It is convenient to introduce the following parametriza- 
tion of the general PDF: 



p(w) 



■y— 1 

-it) ' exp 



-E 



(15) 



with 7 — 1 = — A„ +1 and 1/Z = exp(l — Ao). The param- 
eters Z, 7, Xk are fixed by Eqs and 0. We should 
have Afc > when is the largest exponent, as well as 
when oik is the least negative exponent. 
In Eq. (fl"5|) . despite the condition < 0, with Afc > 0, is 
mathematically acceptable, we should exclude this possi- 
bility from the physical point of view. Because in several 
situations, like with collisional plasmas, such a condition 
would provide a very reduced population at low energy. 
Similarly, also the alternative au > 1, with Afc > 0, for 
some value of fc, should be discarded. This because such 
a condition would give rise to a sub-Maxwellian distribu- 
tion function when, on the contrary, it is expected that 
the high energy tail of the PDF should be larger than the 
one predicted by the Maxwellian distribution (consider, 
for instance, the case of burning plasmas). 
Conjecture (2). Physical considerations lead us to se- 
lect max(ai, ■ • • , a n = 1, and 0),afc < 1 if fc > 2. 
Even though we accept the condition otk < 0, the case 
afc — > 0, should anyhow be considered as a limit situa- 
tion because the graph of p(w) changes abruptly with the 
change of the sign of afc . From this reason, the restriction 
([8]) can be considered as an extreme case. Consequently 
the partial justification of the ansatz, in the Remark (1), 
is given for simplicity and extremality reasons. The ex- 
ploration of the case n = 2, with < ct2 < 1 is ongoing. 



It is clear that in this simplest and extremal case, given 
by the ansatz in the Remark (1), the PDF has the form 



p(w) 



1 



(w/6) 7 " 1 exp(-w/e) (16) 



er( 7 ) 



where r('y) is the Euler Gamma function. The relations 
between the parameters 7, 0, p\ = T/m and v are given 
by 

E(w) = T/m = 78 (17) 
E(logH) = v= (7) + log(9) (18) 

where ^(7) is the digamma function. We draw the at- 
tention to the fact that, in general, the parameters ap- 
pearing in the Eqs and (0) are not completely inde- 
pendent. Indeed suppose that we have E(w ai ) = p\ and 
E(u> Q2 ) = P2 with a\ < 012- Then, by using the Holder 
inequality [27], we obtain: 



u 1/ai 
Mi 



< M 2 



(19) 



In the limit case, by using Eqs (JSJ and (TT2]) , together with 
Jensen's inequality for the exponential function [27| . we 
find 



l/at 

Mfc/ 



> exp (E(log(io))) = cxp(v); k > 1 (20) 



The inequalities (fT9"f and (|2U)) are the necessary condi- 
tions for the existence and not the necessary conditions 
for maximal entropy of the PDF. The inequality (1191) re- 
sults from the constraint whereas (j2T))) from the con- 
straints and 0. In particular, for restrictions (fTT|) 
and (fT2j) we get p\ > exp (v). 

Due to the very special choice of the restrictions used in 
the MaxEnt principle (the minimality of the log(w)), the 
resulting PDF is expected to have some special proper- 
ties. Indeed, the gamma distribution is infinitely divisi- 
ble and stable in the following sense. Let X\, . . . , X n be 
a sequence of n independent random variable having n 
identical gamma distributions (with the same scale pa- 
rameter and shape parameters 71, . . . j n ), then the sum 
^2 k=1 Xk has the same gamma distribution function, 
with the same scale parameter and with shape param- 
eter equal to X\ =1 7k- An important physical property 
of the distributions (TTSl) . in the case < 7 < 1, is the 
following. By varying 7 in this range of parameters, and 
by keeping constant the other parameters, the shape of 
the graph of the DDF changes, due to the singularity 
appearing at w — 0. This singularity can be associated 
with the appearance of a regime of intermittent behavior 
[29l ] . We emphasize that the reference DDF is interpreted 
as time and ensemble average of the physical, DDF. 
We describe a class of stochastic processes admitting 
Eq. (fT5)) as stationary PDF solutions. Note that Eq. (fT5|) 
reduces to the gamma distribution when some coefficients 
are set to zero. To this end, we consider the stochas- 
tic differential equation (SDE) for the random variable 
w(t). In this case w(t) is the energy of an individual 
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charged particle. The SDE includes the simplest soluble 
cases of the class of intermittency models [28jj- [30l] . In 
the Stratonovich version, the SDE reads 

dw(t) = {adt + adB(t)) o w{t) - S [w (<)] dt (21) 

where o stands for the Stratonovich product. In addition, 
a > is the instability threshold, B(t) is the standard 
Brownian motion (Wiener's process) and a is the inten- 
sity of the multiplicative noise. The function S(w) is as- 
sociated with the saturation of the instability controlled 
by the linear term. Hence, we should have 



lim 



S(w) 



w—±oo uj 



(22) 



We also require that the solution near w — is dominated 
by the linear term. So the phenomenology described by 
Eq. (|2ip is still related to the noise-driven intermittency 
if we require that 







(23) 



We will see that, in the particular case S(w) — Aw 2 , the 
stationary solution of Eq. (|2ip is the gamma distribution. 
Hence, the class of Eqs ([2~Tj) includes the generic family of 
equations describing the instability growth, on the pos- 
itive semi- axes (which corresponds to our case), limited 
by the saturation term. We have slightly modified this 
equation by adding the random multiplicative noise term 
adB(t). In the Ito formalism we get 



dw(t) = {adt + adB(t))w(t) -S[w (*)] dt 
with a = a + — 



(24) 



The stationary Fokker-Planck equation for the density 
distribution p(w) reads 



d_ 

dw 



[{a'w — S(w))p(w)] 



d 2 



2 dw 2 



[w 2 p(w)] = 



admitting, up to a normalization constant, the following 
steady state solution 



p(w) = Cw 1 1 exp 

- 2a n 

7= — > 





( S( M 


J 


w z 



(25) 
(26) 



The general form of S(w), compatible with Eqs (f2"2"j) and 
03b, is 



S(w) 



fc=i 



with £,k > 



(27) 



with the constraint that at infinity the coefficient of 
the leading term in Eq. p7[) should be positive. From 
Eqs ([23 and (27]) we obtain 



p(w) = Cw J 1 exp 



m . 



(28) 



By comparing Eq. (f28|) with Eq. (|T5j) . we can link the 
exponents ak with the exponents appearing in Eq. (|27|l . 
Notice that, by setting the parameters entering in the 
Eqs (UJ) and (J27J) as 



7 



2a 

7t2 



7 



and 



Q'k 



6 



(29) 



we obtain, by intermittence mechanism, exactly the same 
stationary distribution function derived by the MaxEnt 
principle under the scale invariant restrictions. In ad- 
dition, we have = A^/^k- Let us now consider the 
standard Landau type generic form for modeling the in- 
stability growth. In the particular case of the simplest 
choice S(w) = Aw 2 , we obtain the PDF from Eq.© 
resulting from the Ansatz in the Remark (1). The high 
energy tail is in agreement with the Maxwell distribution. 
However, for the case of the Landau term S(w) — Aw 3 , 
we obtain a distorted Gaussian distribution 



p(w) — Cw 1 



exp 



(~Aw 2 ) 



(30) 



where the high energy tail decreases too quickly. This 
behaviour is incompatible with many physical situations 
(such as, for example, the case of collisional plasmas) 
and, for this reason, this possibility should be discarded. 
Instead, in accordance with Conjecture (2), if in the lead- 
ing terms A K w 1+ ^ K , we have £i = 1 arj d for the rest 
< £fc < 1, the high energy tail would decrease in a good 
way. Hence, Conjecture (2) is supported by the inter- 
mittency models. Notice that in the special case n = 0, 
in Eqs. ([T5| [28)) the resulting DDF is non-integrable, but 
self similar, like in the case of self organized criticality 
(SOC) models [Uj], [H|. So restriction given by Eq. © 
is a remnant SOC, avalanche-like behavior. By replac- 
ing the self similarity of the DDF by scale invariance 
of the constraints in the MaxEnt principle, we obtain a 
class of DDF that better par ametrize the intermittent, 
avalanche- like behaviour [3l|, [33| of the physical sys- 
tems that can be modeled by self organized criticality 
models [H. 

We conclude this brief report by summarizing the main 
results. Using statistical thermodynamics, we derive the 
general expression of the reference PDF, J-q, by impos- 
ing the minimum entropy production, and the maximum 
entropy principle under the scale invariance restrictions. 
Some restrictions on the parameters entering in J-" are 
obtained by physical arguments. The obtained PDF ex- 
hibits a singularity, which has immediate physical inter- 
pretation in terms of the intermittency models. Finally, 
we derived a family of stochastic processes admitting J-q 
as stationary PDF solutions. 

This work gives several perspectives. Through the ther- 
modynamical field theory (TFT) 35] it is possible to es- 
timate the PDF when the nonlinear contributions cannot 
be neglected [3(|. The next task should be to establish 
the relation between the reference PDF herein derived 
with the one found by the TFT. The solution of this dif- 
ficult problem will contribute to provide a link between 
a microscopic description and a macroscopic approach 



■5 



(TFT). Another problem to be solved is the possibility 
to improve the numerical fit by adding new free param- 
eters according to the principles exposed in this report. 
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